What is a Question? Crowdsourcing Tweet Categorization
نویسندگان
چکیده
One major way in which Amazon Mechanical Turk has been used is in the human labeling (or coding) of data, such as the relevance of search results or quality of Wikipedia articles. Recently, we used Amazon Mechanical Turk for classifying or labeling Twitter updates as questions or not. We present the design of our study and the steps that we took to address the challenges we faced in using Mechanical Turk for this labeling task. We also present our findings and some lessons learnt about the utility and effectiveness of using micro-task markets for conducting large-scale studies involving human-intelligence tasks. Author
منابع مشابه
A Framework for Policy Crowdsourcing
What is the state of the literature in respect to Crowdsourcing for policy making? This work attempts to answer this question by collecting, categorizing, and situating the extant research investigating Crowdsourcing for policy, within the broader Crowdsourcing literature. To do so, the work first extends the Crowdsourcing literature by introducing, defining, explaining, and using seven univers...
متن کاملLinguistically Informed Tweet Categorization for Online Reputation Management
Determining relevant content automatically is a challenging task for any aggregation system. In the business intelligence domain, particularly in the application area of Online Reputation Management, it may be desirable to label tweets as either customer comments which deserve rapid attention or tweets from industry experts or sources regarding the higher-level operations of a particular entity...
متن کاملThe Fundamentals of Policy Crowdsourcing
What is the state of the research on crowdsourcing for policy making? This article begins to answer this question by collecting, categorizing, and situating an extensive body of the extant research investigating policy crowdsourcing, within a new framework built on fundamental typologies from each field. We first define seven universal characteristics of the three general crowdsourcing techniqu...
متن کاملDynamic Allocation of Crowd Contributions for Sentiment Analysis during the 2016 U.S. Presidential Election
Opinions about the 2016 U.S. Presidential Candidates have been expressed in millions of tweets that are challenging to analyze automatically. Crowdsourcing the analysis of political tweets effectively is also difficult, due to large inter-rater disagreements when sarcasm is involved. Each tweet is typically analyzed by a fixed number of workers and majority voting. We here propose a crowdsourci...
متن کاملIdentifying Tweets with Implicit Entity Mentions
ALEX, ADARSH. M.S., Department of Computer Science and Engineering, Wright State University, 2016. Identifying Tweets with Implicit Entity Mentions Social networking sites like Twitter and Facebook have become a significant source of user-generated content in the past decade. Mining of this user-generated content has proved beneficial for a broad range of applications like Event Extraction, Doc...
متن کامل